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No “Earth” in title? 
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“Men Are from Mars, Women are from Venus” 
- John Gray 
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Are data from Mars? 


Why can’t | use Earth tools? 
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Are tools from Venus? 
Why can't | open Earth data? 


Send Feedback 
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Excel 


lap based on VALUE (longitude) and Value. Color shows sum of VALUE (soil moisture). The view is filtered on sum of VALUE (soil moisture), which ranges from 0.0000 to 0.8668. 
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Data Producers from Mars 


¢ | want my data slim and efficient. 
¢ How can | save money in managing data? 


Tool Developers from Venus 


¢ | make my tool work for popular data first. 
¢ Can | make money by supporting your data? 
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Result: Frustrated Users 


@ EOSDIS 


We (HDFEOS.org) can help. 
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We identify gaps in File Formats 


Hierarchical Data Format (HDF) 
Network Common Data Format (netCDF) 


Geospatial Tagged Image File Format 
(Geo TIFF) 


Keyhole Markup Language (KML) / 
Zipped KML (KMZ) 


¢ Comma-separated values (CSV) 
° eIc. 


¢ EOSDIS 8 


We identify gaps in Libraries 


¢ hdf 

¢ netcdf-C 

netcdtf-Java 

¢ HDF — Earth Observing System (hdf-eos) 


Climate and Forecast Metadata (CF) 
conventions 


Geospatial Data Abstraction Library (GDAL) 
= -Clc. 
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We identify gaps in Jools 


¢ Microsoft Excel 

¢ Esri ArcGIS 

¢ Google Earth 

¢ MATLAB 

¢ Python 

¢ Interactive Data Language (IDL) 
¢ Panoply 

¢ Integrated Data Viewer (IDV) 
¢ HDFView 

¢ hddump 

* IG. 
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We identify gaps in Services 


¢ Open-source Project for a Network Data 
Access Protocol (OPeNDAP) 


¢ Web Map Service (WMS) 

¢ Web Map Tile Service (WMTS) 
¢ Web Coverage Service (WCS) 
° ec. 
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AND we provide Solutions... 


¢ File conversion 

¢ Libraries and tools usage 

¢ NASA HDF product specific examples 

¢ Demo services (e.g., Hyrax*, THREDDS**) 
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Suggestions for data producers 


Make HDF5 data work with 

— GDAL 

— netCDF 

~ Hyrax/THREDDS 

Don't forget a few key CF conventions. 
Follow DIWG* recommendations. 
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Suggestions for tool developers 


¢ Download and test NASA HDF products. 
¢ Support them natively. 


¢ Support augmentation. 
— VRT* in GDAL 
— NcML** in netCDF 


¢ Support 3D visualization for data in the air. 
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Suggestions for end-users 


¢ Try OPeNDAP first. CSV may be enough. 
¢ Try netCDF conversion / augmentation. 

¢ Correct metadata with NcML/ VRT. 

¢ Try GEE* instead of GDAL. 

¢ Use CMR wisely. 
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How about Big (fast) data? 


¢ Hadoop / Spark (streaming) / Dask 
¢ Parquet / Arrow 
¢ Elastic Search / Kibana 
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Future: (Deep) Machine Learning? 


¢ scikit-learn / keras / h2o.al 
¢ Please contact us at 


eoshelo@hdfgroup.org if you'd like to see 


examples on machine learning. 
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